Combining Multiple Feature Selection Methods

This paper proposes a feature selection method that combines various feature selection techniques. Feature selection has been realized as one of the most important processes in various applications, especially pattern classification problems. When too many attributes are involved, training a machine to classify patterns into their respective classes is seemingly impossible. Hence, selecting good features is necessary. Albeit numerous methods to select features have been proposed, there exists no universal solution for this problem unless one searches all possible subsets of all attributes. Some techniques such as forward selection and backward elimination are feasible in terms of speed, but suffer from the effect of local optima problem. Exhaustive search technique guarantees to find the optimal subset, but it takes too long for users to wait for the output; its computational time complexity is exponential. Hence, we propose first to reduce the number of features to the minimum size, so that exhaustive search technique can handle in reasonable time, using forward selection and backward elimination techniques. In this way, the selected feature set is much better than those from forward selection and backward elimination and computed much faster than the exhaustive search technique. The proposed combined feature selection technique is tested on the off-line signature verification data set.